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Abstract — Greedy Pursuits are very popular in Compressed 
Sensing for sparse signal recovery. Tliougli many of the Greedy 
Pursuits possess elegant tlieoretical guarantees for performance, 
it is well known that their performance depends on the statistical 
distribution of the non-zero elements in the sparse signal. In 
practice, the distribution of the sparse signal may not be known 
a priori. It is also observed that performance of Greedy Pursuits 
degrades as the number of available measurements decreases 
from a threshold value which is method dependent. To improve 
the performance in these situations, we introduce a novel fusion 
framework for Greedy Pursuits and also propose two algorithms 
for sparse recovery. Through Monte Carlo simulations we show 
that the proposed schemes improve sparse signal recovery in 
clean as well as noisy measurement cases. 

Index Terms — Compressed sensing. Sparse Recovery, Greedy 
Pursuits, Fusion 

I. Introduction 

Compressed Sensing (CS) fl\, fT] uses sparsity of the 
signal to reduce the number of measurements required to 
represent the signal. In general, reconstruction of the signal 
from the compressed measurements is NP-hard. In literature, 
a variety of suboptimal solutions in polynomial time have been 
proposed for this purpose. They can be broadly classified as 
Convex Relaxation methods [fJI, El, Bayesian framework (Sj, 
ill, and Greedy Pursuits (GP) ^-M- 

We consider only GP in this paper GP iteratively es- 
timate the support-set by selecting one or more atoms in 
each iteration, until a convergence criteria is met. In general, 
each iteration of GP consists of two steps: atom(s) selection 
step , and residual updating step. Popular examples of GP 
include Matching Pursuit (MP) |7|, Orthogonal Matching Pur- 
suit (OMP) 1 81, Subspace Pursuit (SP) |9|, and Compressive 
SampHng Matching Pursuit (CoSaMP) [101 . 

Notations : Bold upper case and bold lower case Roman letters denote 
matrices and vectors respectively. Calligraphic letters and Upper case Greek 
alphabets are used to denote sets. represents the p"^ -norm. Aj- denote 
the column sub-matrix of A formed by the columns of A hsted in the set 
T. X7- denote the sub-vector formed by the elements of x whose indices 
are listed in the set 7". T'^ denotes the complement of the set T w.rt. the 
set {1, 2, . . . , N}. For a set T, \T\ denotes its cardinality (size), and for a 
scalar c, |c| denotes the magnitude of c. A-^, and At denote the transpose, 
and pseudo inverse of the matrix A respectively. E denotes the expectation 
operator which is approximated by the sample mean taken over a large number 
of trials. 



Empirically, it has been observed that the recovery per- 
formance of the GP varies and depends on the nature of 
the sparse signal IJ U . [il2J . For e.g., OMP may perform 
better than SP or vice versa for some types of signals. If 
the underlying statistical distribution of the non-zero values 
of the signal is known a priori, we can use the best recovery 
algorithm suitable for that type of signal. But in many practical 
situations, we may not have this prior knowledge and hence, 
we cannot use the best method as the best method is signal 
dependent. 

It can be also seen that any greedy pursuit algorithm requires 
a minimum number of measurements, which is method depen- 
dent, for sparse signal recovery. Also, all GP perform poorly 
in very low dimension measurement regimes. But often many 
applications provide very less number of measurements and 
hence lower dimension measurement regimes are particularly 
important in reality. 

To address these issues, we propose a novel fusion frame- 
work which fuses the information from two GP and estimates 
the correct atoms from the union of the support-sets of the 
two which we refer to as Fusion of Greedy Pursuits (FuGP). 
In this paper, we use OMP and SP as two examples of GP, 
but they can be replaced with any other GP. 

II. CS FRAMEWORK AND GREEDY PURSUITS 

Consider the standard signal acquisition model which ac- 
quires a signal x G via linear measurements using 



Ax + w 



(1) 



where A G M*^^^ represents a measurement (sensing) ma- 
trix, b G M*^ represents the measurement vector, and w G M*^ 
represents the additive measurement noise in the system. In 
CS framework, we have, M <C iV and ([TJ is a well known 
ill-posed problem. But with the additional knowledge that the 
signal is A'-sparseQ (K < M), we can uniquely recover the 
signal under a few assumptions |[T], lIJ]- 

OMP |8| selects one prominent atom which gives the 
maximum correlation value between the columns of A and 

'A signal is said to be X-sparse if at most K of its elements are non-zeros. 



the regularized measurement vector in every iteration. OMP 
finds support-set of a if-sparse signal in K iterations. 

SP |9J first selects the prominent K columns of A from a 
matched filter output. In subsequent steps of the iterations, SP 
refines the initial solution by performing a test for subsets of 
K columns in a group, and maintains a list of K columns of 
A. The refinement of the solution set continues as long as the 
Z2-norm of the residue decreases. 

III. Fusion Framework for GP 

We will start with an experiment which shows the motiva- 
tion of the proposed fusion framework and its significance in 
sparse recovery. 

Consider a CS system where the signal dimension is 500, 
and the sparsity level is 20. Using our notations, we have, 
N — 500 and K = 20. In this example, let us assume that 
the signal is a Gaussian sparse signal in a clean measurement 
setup (see Section |IV] for more details about the simulation 
setup). We consider two CS sparse recovery algorithms viz. 
OMP and SP for reconstruction of the signal. Let T denotes 
the actual support-set, and Tomp and Tsp denote the support- 
sets estimated by OMP and SP respectively. Let Tq^^p — 
T n Tom p and T^p"^ = T (^Tsp represent the sets of true 
atoms estimated by OMP and SP respectively. Then, we have, 
\T\ = \ foMp\ = \ fsp\ ^ K,0 < \fSTip\ < K, and < 

Ifjp"'^! < K. 

Table I] presents the average (over 10, 000 trials) number of 
true atoms in the estimated support-sets for Gaussian sparse 
signals for different values of a where a is defined as 

a = M/N. (2) 

a denote the fraction of measurements, also called the normal- 
ized measure of problem indeterminacy [ 12 1. The details of the 
simulation are given in Section lTV-BI It can be seen from Table 
I] that for a = 0.12, the average number of correctly estimated 
atoms by OMP is 8.1, and by SP is 10.5. Interestingly, the 
average number of correct atoms in the union of the support- 
sets estimated by OMP and SP is 12.4, closer to the true value 
20. Also, using the property of union operator in set theory, 
it is guaranteed that the union of the estimated support-sets 
always contain at least as many true atoms as in the estimated 
support-set of the best performing algorithm. 
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TABLE I 

Average number of correctly estimated atoms by OMP and SP, 
FOR Gaussian sparse signal, in clean measurement case, 

AVERAGED OVER 10, 000 TRIALS (TV = 500, K = 20). 



among the atoms in the union set is (^^) ((20) ™ 
experiment) in the worst case, which is significantly small 
as compared to the original where it is (^) ((^2™) °™ 
experiment). But for larger values of K, {^^) is still very 
large. Hence, by employing some efficient scheme to select 
K atoms from 2K atoms, we may be able to improve the 
number of correctly estimated atoms and improve the quality 
of the reconstructed sparse signal. 

A. Proposed Fusion Framework 

To develop the fusion framework using OMP and SP as 
ingredient algorithms, let us call the union of the estimated 
support-sets as joint support-set and denote by F = Tomp U 
Tsp ■ Also, we call the intersection of the estimated support- 
sets as common support-set and denote by A = Tomp (^Tsp- 
We have, \foMp\ = \fsp\ = K, < \A\ < K and K < 
|F| < 2K. In the fusion framework, our task is to pick K 
atoms from the joint support-set with |r| atoms. Assuming 
M > 2K, we propose a least-squares based method in FuGP 
for this purpose. 

Now, let us define two algorithmic functions which will be 
used in the proposed sparse recovery algorithms. 

Definition. Let A G M*^^^, b e M^-^^^ and K be the 
sparsity level. Also let Tmit denote the initial support-set 
with \Tnit\ < K. Then we define the following algorithmic 
functions. 

foMP = OMP{A,h,K,Tmt) 
fsp^ SPiA,h,K,Tmt) 

where \ foMp \ = \ fsp\ = K. 

The functions ''OMP" and execute Algorithms [T| and 
Irrespectively. Note that by putting Tmit = in Algorithms 
[U and |2] we get classic OMP and SP respectively. 



Algoritlim 1 OMP with Initial Support 
Inputs: Amxn, ^mxi, K, and Tnit 
Ensure: \Tinit \ <K~1 

1: fc = \Tinit\; 

2: rk=h AT„,it A^^^^^b; 

3: Tk — Tinitt 

4: repeat 

5: + 

6: ik = argmax afr^.i; 

l=l:N,l^fk-i 
7: fk^lkUfk-U 

8: Tk ^h- Af^A^^h; 
9: until (k > K); 

10: r = Tfe; 

Outputs: T, Xj- = At b, and = 0. 



These observations lead to the possibility of estimating 
more correct atoms from the union set than that individually 
estimated by OMP and SP algorithms. The exhaustive search 



Since both methods (OMP and SP) agree on the atoms 
selected in A, we give more confidence on these atoms as 
compared to any other atom in F. Hence FuGP includes these 



Algorithm 2 SP with Initial Support 



Inputs: Amxn, ^mxi, K, and Ttmt 
Ensure: \%mt \ <K-1 
1 

2: 
3: 
4: 
5: 



b; 



fc = 0; 

rk = h-AT, 

repeat 

k = k+\; 
J = indices of the K highest ampHtude 
components of A^rfc_ 

r =Jufk-i; 



-1, 



0; 



Tk = indices corresponding to the K largest 

magnitude entries in v; 
Tfc = b 
until (||rfe||2 > \\rk- 



A^. Al b; 

-11I2); 



Outputs: r, = At b. 



and x^^ = 0. 



atoms in the solution set. Now, we need to identify only 
— |A| atoms from the remaining |r| — |A| atoms. Applying 
least-squares on the atoms in F, we form an intermediary 
solution for the signal. The remaining K —\A\ support atoms, 
we denote by T, are estimated as the K — |A| indices 
corresponding to the largest magnitudes in v which are not 
in A. Now, the support-set is estimated as the union of the 
atoms in the sets T and A. Finally, the sparse signal estimate is 
found from the estimated support-set using the least-squares. 
The main steps of the FuGP algorithm for fusing estimated 
support-sets of OMP and SP are summarized in Algorithm |3] 



Remarks: 

m The fusion framework and FuGP algorithm are scalable 
and can be easily extended to accommodate more than 
two greedy pursuit algorithms. 
« The performance of FuGP directly depends on the num- 
ber of correct atoms in the joint support-set. Hence, we 
should choose the underlying algorithms such that join 
support-set contains maximum number of correct atoms. 
Iterative Fusion: Proposed FuGP aims to identify the correct 
atoms present the joint support-set. Hence it can at most 
identify all the correct atoms in the joint support-set. But in 
lower dimensional measurement regimes, the joint support- set 
may not contain all correct atoms and FuGP will surely miss 
the correct atoms which are not included in the joint support - 
set. To address this issue and hence improve the performance, 
we propose an iterative version of the fusion algorithm called 
Iterative Fusion of Greedy Pursuits (IFuGP). 

In fc* iteration, we find the common support-set A^ and call 
OMP and SP with A^ as the initial support-set to identify the 
remaining atoms in the support. Aq is initialized as null set. 
Afc is updated as the common atoms in the newly estimated 
support-sets by OMP and SP. We fuse the support-sets newly 
estimated by OMP and SP to find the new estimate of the 
support-set. The iteration continues as long as the /2-norm of 
the residue decreases. This procedure opens a possibility to 
include more correct atoms in the joint support-set Fj. which 
are not in the previous iteration k — 1. It may observed that 
we internally call FuGP in each iteration. 

IFuGP algorithm is explained in Algorithm |4] Observe 
that Steps l6lfT3] in Algorithm |4] essentially forms the FuGP 
algorithm. IFuGP is computationally more demanding than 
FuGP 



Algorithm 3 FuGP Algorithm 



Algorithm 4 IFuGP Algorithm 



Inputs: Amxn, ^mxi, and K 



ToMP^OMP{A,h,K,i 
fsp = SPiA,h,K,0y, 
A = ToMP n Tsp; 
r = To MP u Tsp\ 

vrc = 0; 



vr = A^h, 



); ► Using Algorithm [T] 

► Using Algorithmic] 
► (0 < |A| < K) 
*■ (K < \T\ < 2K) 
► intermediary signal estimate 



T = indices corresponding to the (K — |A|) largest 
magnitude entries in v which are not in A; 

r = ru A; 



Output: T, Xj- — Aljjb, and x^^ 



0. 



The computational demand of FuGP algorithm is a little 
more than the added computational cost of the two underlying 
methods. For e.g., the computational complexity of both OMP 
and SP are 0{MNK) independently. The computational 
complexity of FuGP in this case remains 0{MNK). To save 
the running time, we can run both SP and OMP in parallel 
and then apply FuGP on the estimated support sets. It may 
be also observed that the memory requirement of core part of 
FuGP (steps [3ll6l in Algorithm IS is only 0{N). 



Inputs: A 



MxN, Oj\/xl' 



and K 



1: fc 0; 

2: ro = b; 

3: Ao = 0; 

4: repeat 

5: k^k + l\ 

6: ToMP = OMP{A, h, K, Afc-i); ► Using Algorithms 
7: fsp = SP{A,h,K,Ak-i); ► Using Algorithmic] 

Ak = ToMpnTsp; >^o<\Ak\<K 
Tfc = ToMP u Tsp; ► if < IFfc I < 27f 

vp^, = Ap^b, v-pc =0; ► intermediary signal estimate 
Tk = indices corresponding to the (K — \ Ak\) largest 

magnitude entries in v which are not in A^; 
fk = fkUAk; >^\fk\^K 
rk=h^ ' At 

until (||rfe||2 
f = fk-i; 
Output: T, Xj- = At^b, and x^^ = 0. 



A^ Al b; 

> llrfe-illa); 



In this paper, for notational brevity, we denote 
FuGP(OMP,SP) and IFuGP(OMP,SP) by FuGP and IFuGP 




respectively. 

IV. Simulation and Results 

We did extensive Monte Carlo simulations to evaluate per- 
formance of the proposed methods. We explain the simulation 
setup and define the performance measure used for comparing 
different methods, in this section. 

A. Performance Measure 

Signal-to-Reconstruction-Error Ratio (SRER): Let x and 
X denote the original and reconstructed sparse signal vector. 
SRER is a performance measure build on top of Mean Square 
Error information. SRER (in dB) is defined as 

Fllxll^ 

SRER (in dB) ^ 10 logio ^ (3) 

1B^||X X||2 

Signal to Measurement-Noise Ratio (SMNR):: Let and 
(T^j denote the power of each element of signal and noise vector 
respectively. For noisy measurement simulations, we define 
SMNR in dB as 

SMNR (in dB)^ mo,,, ^^^lOlo,,,^^. (4) 

B. Experimental Setup 

Many of the GP including OMP and SP provide theoretical 
guarantees for convergence, but the theoretical bounds are 
by and large "pessimistic" worst case bounds. In general, all 
CS sparse recovery methods work efficiently near this region. 
But in many applications, the number of measurements may 
be very limited due to many practical reasons. Hence we 
are more interested in the lower dimensional measurement 
regimes where the sparse recovery methods begin to collapse. 
To compare the performance of the proposed methods with 
OMP and SP in this highly under-sampled region, we choose 
small values of a where a is defined in (|2]l. 
The main steps involved in the simulation are the following: 

1) Fix K, N and choose an a so that the number of 
measurements M is an integer. 



2) Generate elements of Amxn independently from 
JV{0, jj) and normalize each column norm to unity. 

3) Choose K locations uniformly over the set {1,2. ..N} 
and fill these locations of x based on the choice of signal 
characteristics: 

a) Gaussian sparse signal: non-zero values indepen- 
dently from 7V(0, 1). 

b) Rademacher sparse signal: non-zero values are set 
to H-1 or -1 with probabiUty ^. They are also known 
as "constant amplitude random sign" signals. 

Set remaining N ~ K locations of x as zeros. 

4) For noisy regime, the additive noise vif is a Gaussian 
random vector whose elements are independently chosen 
from 7\/'(0, a^) and for clean case, w is set to zero. 

5) The measurement vector b = Ax + w. 

6) Apply the reconstruction methods independently. 

7) Repeat steps [3]|6] T times. T indicates the number of 
times X independently generated, for a fixed A. 

8) Repeat steps |2]|7] S times. S indicates the number of 
times A is independently generated. 

9) Calculate SRER by averaging over S x T data 
10) Repeat steps |2l|9] for different values of a. 

C. Results and Discussions 

We performed Monte Carlo simulation with following pa- 
rameters. N = 500, K ^ 20, S = 100, T = 100. i.e., we 
generated the measurement matrix A 100 times and for each 
realization of A, we generated a sparse signal with ambient 
dimension 500 and sparsity level K ~ 20, 100 times. 

Gaussian Sparse Signal: The performance of FuGP and 
IFuGP with OMP and SP as ingredient methods for Gaussian 
sparse signals in clean as well as noisy measurement cases 
are shown in Fig. [T] FuGP showed a significant improvement 
as compared to the ingredient methods OMP and SP in both 
cases. IFuGP was able to give an improvement over FuGP. For 
example, in Fig.IIla), for a = 0.18, FuGP gave 6.5 dB (31%) 
and 10 dB (58%) SRER improvement respectively over OMP 
and SP in clean measurement case. For the same scenario. 
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(a) Clean Measurement (b) Noisy Measurement (SMNR = 15 dB) 

Fig. 2. Rademacher sparse signals: Signal-to-Reconstruction-Error Ratio (SRER) vs. Fraction of Measurements (A^ = 500, K = 20) 



IFuGP(OMP, SP) was able to improve the performance further 
and showed 12 dB (59%), and 16 dB (91%) improvement 
respectively over OMP and SP. Compared to FuGP, IFuGP 
gave 5.72 dB (21%) additional performance improvement in 
SRER. 

In the noisy measurement case (refer Fig. [Tib)), for a — 
0.18, FuGP improved the performance by 1.1 dB (11%) 
and 3.1 dB (36%) over OMP and SP respectively. For the 
same situation, IFuGP gave 1.5 dB (13%), and 3.5 dB (40%) 
additional SRER as compared to OMP and SP respectively, 
and also showed 0.35 dB (3%) SRER improvement over FuGP. 

Rademacher Sparse Signal: The results of simulation for 
Rademacher sparse signal is shown in Fig. |2] Here also FuGP 
and IFuGP showed performance improvement over OMP and 
SP. In the clean measurement case (refer Fig. EJa)), for a = 
0.25, FuGP gave 18 dB (203%) and 2.8 dB (12%) SRER 
improvement that OMP and SP respectively. IFuGP further 
improved the performance and gave SRER improvement by 
22dB (252%) and 7 dB (30%) respectively over OMP and SP 
In this case IFuGP showed 4.3 dB (16%) SRER improvement 
over FuGP. 

In the noisy measurement simulation (refer Fig. EJb)), for 
a = 0.25, by employing FuGP, an additional SRER improve- 
ment of 11.5 dB (172%) and 1.1 dB (6%) was achieved as 
compared to OMP and SP respectively. In this case also, 
IFuGP continued to improve the performance over FuGP (1.7 
dB (9%) SRER improvement than FuGP) resulting in 13.2 dB 
(196%) and 2.7 dB (16%) SRER improvement over OMP and 
SP respectively. 

From the simulation results, it can be seen that FuGP and 
IFuGP improved the sparse signal recovery consistently in 
all the cases as compared to the ingredient methods. The 
robustness against noise was shown in noisy measurement 
simulations for an SMNR = 15 dB, which closely matches 
many application scenarios. 

Reproducible Results: In the spirit of repro- 
ducible research, we provide necessary Matlab 
codes downloadable in the following website: 



|http : / / www . ece . iisc . ernet .in/ ~sspiab/ Public /FuGP . t 
The code reproduces the simulation results shown in Fig. [T] 
and Fig. |2] 

V. Conclusions 

We proposed a novel fusion framework for Greedy Pursuits 
and also proposed two algorithms to recover the sparse signals. 
Using simulations we showed that the proposed scheme can 
improve the sparse signal recovery performance of Greedy 
Pursuits in clean as well as noisy measurement cases. 
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